A Tutorial of Adversarial Learning

Just for fun, far from complete. 1 Problem Definition We define x ∈ X as an instance, where X is the instance space. x is represented by a vector variable with n dimensions, namely x = (x1, . . . , xn). xi denotes the ith feature in instance x. Each instance can belong to one of two classes: positive (malicious) or negative (innocent), which are denoted by x and x respectively. Let training set S ⊂ X and test set T ⊂ X consist of both positive and negative instances. In practice, S is usually a finite set, whereas T = X \ S has infinite size. We call a function C : X 7→ {−1, 1} as a Boolean classifier, or a classifier for short. We refer to x for which C(x) = 1 and x for which C(x) = −1. The task of a classifier is to learn from S a function C(x) that will correctly predict new instance x ∈ T . Obviously, a well-performed classifier can secure the system by detecting malicious instances in advance (e.g. spam filtering and virus detection). An adversary attempts to defeat the system by sending malicious instances without being detected. In fact, adversaries are actively disguising their behavior to evade detection. For instance, senders of junk email often add “good” words or sentences to cheat the spam filter for decreasing the likelihood of detection. Although these disguised instances are more indicative of innocent than malicious, they may also decrease the reward of adversary due to their ineffectiveness. For adversary, some instances are more effective than others. We explain such differences on utility by an adversarial cost function A(x) 7→ R. Note, that A(x) is domain-dependent function. We assume that adversaries have a base instance x for which C(x) = 1 on hand. To evade detection, they are interested in finding an instance x ∗ that most similar to x but will be classified as negative. To measure the similarity between two instances, we first define an adversarial cost function as A(x) = n ∑ i=1 ai|xi − x a i |, where positive scalars1 ai represent the relative cost of changing each feature, allowing that some features may be more important than others (from adversaries’ perspective of view). An illustrative example is depicted in Figure 1. 1 The positive scalar ai is under an assumption that x is the best instance as far as the adversary knows. That is, any changes to x costs an utility loss.